1. Field of the Invention
Embodiments of the present invention relate to a method and apparatus for estimating a disparity for 3D object recognition, and more particularly, to a method of estimating a disparity for 3D object recognition from a stereo image picked up with two cameras separated from each other by a predetermined distance.
2. Description of the Related Art
Stereo matching is a process for extracting depth information from two images of the same scene picked up at different viewpoints. The depth information is calculated based on relative positions of the same object projected on the two images. In the stereo matching process, it is difficult to point out pixels representing the same object in the two images.
Positions of the corresponding pixels in the two images are related to a one-dimensional variation in the epi-polar line direction, that is, a disparity for the images. A disparity is estimated by locating a pixel of the one image relative to the corresponding pixel of the other image. The estimation of the disparity presents an energy minimization problem. In order to solve the energy minimization problem, there are proposed various algorithms including a graph cut algorithm.
In the graph cut algorithm, the estimation of the disparity begins with obtaining a disparity map f which minimizes an energy E(f) represented with Equation 1,E(f)=Edata(f)+Esmooth(f),  (1)where Edata(f) denotes a data energy obtained by measuring disagreement between the corresponding pixels of the stereo image, and Esmooth(f) denotes a smooth energy obtained by measuring smoothness of the disparity map.
In the graph cut algorithm, the data energy Edata(f) is calculated from a difference between pixel values (or intensities) corresponding to the same real object in the two images. The smooth energy Esmooth(f) is calculated by using Equation 2,
                                                        E              smooth                        ⁡                          (              f              )                                =                                    ∑                                                {                                      p                    ,                    q                                    }                                ⁢                EN                                      ⁢                                                            V                                      p                    ,                    q                                                  ⁡                                  (                                                            f                      p                                        ,                                          f                      q                                                        )                                            ⁢                              T                ⁡                                  (                                                            f                      ⁡                                              (                        p                        )                                                              =                                          f                      ⁡                                              (                        q                        )                                                                              )                                                                    ⁢                                  ⁢                              T            ⁡                          (              a              )                                =                      {                                                                                                                              0                        ⁢                                                                                                  ⁢                        a                        ⁢                                                                                                  ⁢                        is                        ⁢                                                                                                  ⁢                        false                                                                                                                                                1                        ⁢                                                                                                  ⁢                        a                        ⁢                                                                                                  ⁢                        is                        ⁢                                                                                                  ⁢                        true                                                                                            ⁢                                                                  ⁢                                                      V                                          p                      ,                      q                                                        ⁡                                      (                                                                  f                        p                                            ,                                              f                        q                                                              )                                                              =                              {                                                                                                    0                                                                                                                          f                            p                                                    =                                                      f                            q                                                                                                                                                              λ                                                                                                                                                        (                                                                                                f                                  p                                                                ≠                                                                  f                                  q                                                                                            )                                                        &                                                    ⁢                                                                                                          ⁢                                                      (                                                                                                                                                                                                                                      I                                      L                                                                        ⁡                                                                          (                                      p                                      )                                                                                                        -                                                                                                            I                                      L                                                                        ⁡                                                                          (                                      q                                      )                                                                                                                                                                                                  >                              5                                                        )                                                                                                                                                                                        2                          ⁢                          λ                                                                                                                                                                                (                                                                                                f                                  p                                                                ≠                                                                  f                                  q                                                                                            )                                                        &                                                    ⁢                                                                                                          ⁢                                                      (                                                                                                                                                                                                                                      I                                      L                                                                        ⁡                                                                          (                                      p                                      )                                                                                                        -                                                                                                            I                                      L                                                                        ⁡                                                                          (                                      q                                      )                                                                                                                                                                                                  ≤                              5                                                        )                                                                                                                                ,                                                                                        (        2        )            where p and q denote neighboring pixels in one image, and f(p) and f(q) denote disparities for the pixels p and q, respectively. N denotes a set of neighboring pixels p and q. Vp,q(fp, fq) denotes a penalty. If disparities for the two neighboring pixels p and q are different from each other, the penalty Vp,q(fp, fq) increases the smooth energy as described in Equation 2. λ denotes a smooth energy coefficient.
In the graph cut algorithm for estimating the disparity, how the smooth energy coefficient λ is determined is important.
FIGS. 1 and 2 show original images and associated disparity maps. More specifically, FIGS. 1A and 1B show left and right original images, respectively. FIG. 1C shows a ground-truth disparity map. FIGS. 2A to 2C show disparity maps for the smooth energy coefficients λ of 2, 6, and 12, respectively, obtained by using the graph cut algorithm. FIG. 2A shows a disparity map of an under-smooth case, which contains a large amount of noise. FIG. 2C shows a disparity map of an over-smooth case, which hardly represents discontinuities between the objects. FIG. 2B shows more distinguishable objects than the cases of FIGS. 2A and 2C.
FIG. 3 shows gross error ratios of disparity maps with respect to the ground-truth disparity map according to the smooth energy coefficient λ when an image has different resolutions. As shown in FIG. 3, it can be understood that an optimal smooth energy coefficient λ for a half-resolution image is completely different from an optimal smooth energy coefficient λ for a double-resolution image.
Therefore, in order to obtain an optimal disparity map for a stereo image, it is necessary to calculate different smooth energy coefficients for images.